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Abstract 

The motivation behind mathematically modeling the human operator is to help ex- 
plain the response characteristics of the complex dynamical system including the human 
manual controller. In this paper, we present two approaches to human operator model- 
ing: classical linear control approach and modern nonlinear control approach. The latter 
one is formalized using both fixed and adaptive Lie-Derivative based controllers. 
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1 Introduction 

Despite the increasing trend toward automation, robotics and artificial intelligence (AI) 
in many environments, the human operator will probably continue for some time to 
be integrally involved in the control and regulation of various machines (e.g., missile- 
launchers, ground vehicles, watercrafts, submarines, spacecrafts, helicopters, jet fight- 
ers, etc.). A typical manual control task is the task in which control of these machines is 
accomplished by manipulation of the hands or fingers [1]. As human-computer inter- 
faces evolve, interaction techniques increasingly involve a much more continuous form 
of interaction with the user, over both human-to-computer (input) and computer-to- 
human (output) channels. Such interaction could involve gestures, speech and anima- 
tion in addition to more 'conventional' interaction via mouse, joystick and keyboard. 
This poses a problem for the design of interactive systems as it becomes increasingly 
necessary to consider interactions occurring over an interval, in continuous time. 
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The so-called manual control theory developed out of the efforts of feedback con- 
trol engineers during and after the World War II, who required models of human perfor- 
mance for continuous military tasks, such as tracking with anti-aircraft guns |ED. This 
seems to be an area worth exploring, firstly since it is generally concerned with systems 
which are controlled in continuous time by the user, although discrete time analogues 
of the various models exist. Secondly, it is an approach which models both system and 
user and hence is compatible with research efforts on 'synthetic' models, in which as- 
pects of both system and user are specified within the same framework. Thirdly, it is an 
approach where continuous mathematics is used to describe functions of time. Finally, 
it is a theory which has been validated with respect to experimental data and applied 
extensively within the military domains such as avionics. 

The premise of manual control theory is that for certain tasks, the performance of the 
human operator can be well approximated by a describing function, much as an inani- 
mate controller would be. Hence, in the literature frequency domain representations of 
behavior in continuous time are applied. Two of the main classes of system modelled 
by the theory are compensatory and pursuit systems. A system where only the error sig- 
nal is available to the human operator is a compensatory system. A system where both 
the target and current output are available is called a pursuit system. In many pursuit 
systems the user can also see a portion of the input in advance; such tasks are called 
preview tasks O. 

A simple and widely used model is the 'crossover model' |9), which has two main 
parameters, a gain K and a time delay t, given by the transfer function in the Laplace 
transform s domain 

p-rs 

H = K- . 



s 

Even with this simple model we can investigate some quite interesting phenomena. For 
example consider a compensatory system with a certain delay, if we have a low gain, 
then the system will move only slowly towards the target, and hence will seem sluggish. 
An expanded version of the crossover model is given by the transfer function HI 

H _ K (T L s + l)e-( TS+a M 
(T lS + l)(T N s + l) ' 

where Tl and Tj are the lead and lag constants (which describe the equalization of the 
human operator), while the first-order lag (T^S + 1) approximates the neuromuscular 
lag of the hand and arm. The expanded term a/s in the time delay accounts for the 
'phase drop', i.e., increased lags observed at very low frequency Q. 

Alternatively if the gain K is very high, then the system is very likely to over- 
shoot the target, requiring an adjustment in the opposite direction, which may in turn 
overshoot, and so on. This is known as 'oscillatory behavior'. Many more detailed 
models have also been developed; there are 'anthropomorphic models', which have a 
cognitive or physiological basis. For example the 'structural model' attempts to reflect 
the structure of the human, with central nervous system, neuromuscular and vestibular 
components 0. Alternatively there is the 'optimal control modeling' approach, where 
algorithmic models which very closely match empirical data are used, but which do not 
have any direct relationship or explanation in terms of human neural and cognitive ar- 
chitecture [10]. In this model, an operator is assumed to perceive a vector of displayed 
quantities and must exercise control to minimize a cost functional given by fl} 

J = E{ lim 1 / [q iy f(t) + J> iU 2 (t) + 9i u 2 (t)))dt} , 
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which means that the operator will attempt to minimize the expected value E of some 
weighted combination of squared display error y, squared control displacement u and 
squared control velocity u. The relative values of the weighting constants , , gi will 
depend upon the relative importance of control precision, control effort and fuel expen- 
diture. 

In the case of manual control of a vehicle, this modeling yields the 'closed-loop' or 
'operator-vehicle' dynamics. A quantitative explanation of this closed-loop behavior is 
necessary to summarize operator behavioral data, to understand operator control actions, 
and to predict the operator-vehicle dynamic characteristics. For these reasons, control 
engineering methodologies are applied to modeling human operators. These 'control 
theoretic' models primarily attempt to represent the operator's control behavior, not the 
physiological and psychological structure of the operator (6l|7l. These models 'gain in 
acceptability' if they can identify features of these structures, 'although they cannot be 
rejected' for failing to do so [8]. 

One broad division of human operator models is whether they simulated a contin- 
uous or discontinuous operator control strategy. Significant success has been achieved 
in modeling human operators performing compensatory and pursuit tracking tasks by 
employing continuous, quasi-linear operator models. Examples of these include the 
crossover optimal control models mentioned above. 

Discontinuous input behavior is often observed during manual control of large am- 
plitude and acquisition tasks (^QTICGICE]]. These discontinuous human operator re- 
sponses are usually associated with precognitive human control behavior (HO. Dis- 
continuous control strategies have been previously described by 'bang-bang' or relay 
control techniques. In Ifl5ll . the authors highlighted operator's preference for this type 
of relay control strategy in a study that compared controlling high-order system plants 
with a proportional verses a relay control stick. By allowing the operator to generate a 
sharper step input, the relay control stick improved the operators' performance by up to 
50 percent. These authors hypothesized that when a human controls a high-order plant, 
the operator must consider the error of the system to be dependent upon the integral of 
the control input. Pulse and step inputs would reduce the integration requirements on the 
operator and should make the system error response more predictable to the operator. 

Although operators may employ a bang-bang control strategy, they often impose 
an internal limit on the magnitude of control inputs. This internal limit is typically 
less than the full control authority available [9]. Some authors |[T6l hypothesized that 
this behavior is due to the operator's recognition of their own reaction time delay. The 
operator must tradeoff the cost of a switching time error with the cost of limiting the 
velocity of the output to a value less than the maximum. 

A significant amount of research during the 1960's and 1970's examined discontin- 
uous input behavior by human operators and developed models to emulate it |[T4l[T7llT8l 
[m|20l|2rJl22ll22l24l- Good summaries of these efforts can be found in lIBI ifTTI ll9l 
and OlTl. All of these efforts employed some type of relay element to model the dis- 
continuous input behavior. During the 1980's and 1990's, pilot models were developed 
that included switching or discrete changes in pilot behavior |[^l27ll28l[29l[T2llT3l . 

Recently, the so-called 'variable structure control' techniques were applied to model 
human operator behavior during acquisition tasks (6l 13. The result was a coupled, 
multi-input model replicating the discontinuous control strategy. In this formulation, a 
switching surface was the mathematical representation of the human operator's control 
strategy. The performance of the variable strategy model was evaluated by considering 
the longitudinal control of an aircraft during the visual landing task. 
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In this paper, we present two approaches to human operator modeling: classical 
linear control approach and modern nonlinear Lie-Derivative based control approach. 



2 Classical Control Theory versus Nonlinear Dy- 
namics and Control 

In this section we review classical feedback control theory (see e.g., ll30l 01 [311) and 
contrast it with nonlinear and stochastic dynamics (see e.g., IT321I331I341 ). 

2.1 Basics of Kalman's Linear State-Space Theory 

Linear multiple input-multiple output (MIMO) control systems can always be put into 
Kalman canonical state-space form of order n, with m inputs and k outputs. In the case 
of continual time systems we have state and output equation of the form 

dx/dt = A(i)x(i) +B(i)u(i), (1) 

y(i) = C(t)x(t)+D(t)u(t), 

while in case of discrete time systems we have state and output equation of the form 

x(n + l) = A(n)x(n) + B(ra)u(n), (2) 
y(n) = C(n) x(n) + D(n) u(n). 

Both in (Q]) and in Q the variables have the following meaning: 

x(t) 6 X is an n— vector of state variables belonging to the state space XcK"; 
u(t) E U is an m— vector of inputs belonging to the input space U C W 71 ; 
y(t) E Y is a k— vector of outputs belonging to the output space Y C 



A(t) 
B(t) 
C(i) 
D(i) 



X->Xisannxn matrix of state dynamics; 
U — ► X is an n x m matrix of input map; 
X — > Y is an k x n matrix of output map; 
U^Yisanfexm matrix of input-output transform. 



Input u(i) E U can be empirically determined by trial and error; it is properly 
defined by optimization process called Kalman regulator, or more generally (in the 
presence of noise), by Kalman filter (even better, extended Kalman filter to deal with 
stochastic nonlinearities). 



2.2 Linear Stationary Systems and Operators 

The most common special case of the general Kalman model (Q~|), with constant state, 
input and output matrices (and relaxed boldface vector-matrix notation), is the so-called 
stationary linear model 

x = Ax + Bu, y = Cx. (3) 

The stationary linear system (0 defines a variety of operators, in particular those related 
to the following problems: 

1. regulators, 

2. end point controls, 

3. servomechanisms, and 

4. repetitive modes (see 11391 ). 
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2.2.1 Regulator Problem and the Steady State Operator 

Consider a variable, or set of variables, associated with a dynamical system. They 
are to be maintained at some desired values in the face of changing circumstances. 
There exist a second set of parameters that can be adjusted so as to achieve the desired 
regulation. The effecting variables are usually called inputs and the affected variables 
called outputs. Specific examples include the regulation of the thrust of a jet engine by 
controlling the flow of fuel, as well as the regulation of the oxygen content of the blood 
using the respiratory rate. 

Now, there is the steady state operator of particular relevance for the regulator prob- 
lem. It is 

Uoo = —C A BiLoo, 

which describes the map from constant values of u to the equilibrium value of y. It is 
defined whenever A is invertible but the steady state value will only be achieved by a 
real system if, in addition, the eigenvalues of A have negative real parts. Only when the 
rank of CA~ X B equals the dimension of y can we steer y to an arbittary steady state 
value and hold it there with a constant u. A nonlinear version of this problem plays a 
central role in robotics where it is called the inverse kinematics problem (see, e.g., Il40l0 . 

2.2.2 End Point Control Problem and the Adjustment Operator 

Here we have inputs, outputs and trajectories. In this case the shape of the trajectory is 
not of great concern but rather it is the end point that is of primary importance. Standard 
examples include rendezvous problems such as one has in space exploration. 

Now, the operator of relevance for the end point control problem, is the operator 

x(T)= f exp[A(T - a)} Bu(a) da. 
Jo 

If we consider this to define a map from the rnD L2 space [0, T] (where u takes on 
its values) into R. m then, if it is an onto map, it has a Moore-Penrose (least squares) 
inverse 

u(a) = B T exp[A T (T - a)} (W[0, T])~ l (x(T) - exp(AT) x(0)) , 

with the symmetric positive definite matrix W, the controllability Gramian, being given 
by 

W[0, T]= f exp[A(T - a)] BB T exp[A T (T - a)] da. 
Jo 

2.2.3 Servomechanism Problem and the Corresponding Operator 

Here we have inputs, outputs and trajectories, as above, and an associated dynamical 
system. In this case, however, it is desired to cause the outputs to follow a trajectory 
specified by the input. For example, the control of an airplane so that it will travel along 
the flight path specified by the flight controller. 

Now, because we have assumed that A, B and C are constant 

y{t)=C exp(At) x(0) + [ C exp[A(T - r)] Bu(t) dr, 

Jo 

and, as usual, the Laplace transform can be used to convert convolution to multiplication. 
This brings out the significance of the Laplace transform pair 
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Cexp(At)B C(Is - A)~ 1 B (4) 

as a means of characterizing the input-output map of a linear model with constant coef- 
ficients. 

2.2.4 Repetitive Mode Problem and the Corresponding Operator 

Here again one has some variable, or set of variables, associated with a dynamical sys- 
tem and some inputs which influence its evolution. The task has elements which are 
repetitive and are to be done efficiently. Examples from biology include the control of 
respiratory processes, control of the pumping action of the heart, control of successive 
trials in practicing a athletic event. 

The relevant operator is similar to the servomechanism operator, however the con- 
straint that u and x are periodic means that the relevant diagonalization is provided by 
Fourier series, rather than the Laplace transform. Thus, in the Fourier domain, we are 
interested in a set of complex matrices 

G(iwi) = C(iwi — A)~ 1 B, Wi = 0, wq, 2wq, ... 

More general, but still deterministic, models of the input-state-output relation are 
afforded by the nonlinear affine model (see, e.g., fiTI ) 

x{t) = f(x{t))+g{x(t))u{t), 
y(t) = h(x(t)); 

and the still more general fully nonlinear model 

x(t) = f(x(t),u{t)), 
y(t) = h{x(t)). 

2.2.5 Feedback Changes the Operator 

No idea is more central to automatic control than the idea of feedback. When an input 
is altered on the basis of the difference between the actual output of the system and the 
desired output, the system is said to involve feedback. Man made systems are often 
constructed by starting with a basic element such as a motor, a burner, a grinder, etc. 
and then adding sensors and the hardware necessary to use the measurement generated 
by the sensors to regulate the performance of the basic element. This is the essence of 
feedback control. Feedback is often contrasted with open loop systems in which the 
inputs to the basic element is determined without reference to any measurement of the 
trajectories. When the word feedback is used to describe naturally occurring systems, it 
is usually implicit that the behavior of the system can best be explained by pretending 
that it was designed as one sees man made systems being designed ll39l . 

In the context of linear systems, the effect of feedback is easily described. If we start 
with the stationary linear system © with u being the controls and y being the measured 
quantities, then the effect of feedback is to replace u by u — Ky with K being a matrix 
of feedback gains. The closed-loop equations are then 

x = (A - BKC) x + Bu, y = Cx. 
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Expressed in terms of the Laplace transform pairs (01), feedback effects the transforma- 
tion 

(Cexp(At)B; C{Is - A)' 1 B) i — ► C exp(^ - BKC) l B- C(Is -A + BKC)~ l B. 

Using such a transformation, it is possible to alter the dynamics of a system in a signifi- 
cant way The modifications one can effect by feedback include influencing the location 
of the eigenvalues and consequently the stability of the system. In fact, if K is m by p 
and if we wish to select a gain matrix K so that A—BKC has eigenvalues Ai, A2, A n , 
it is necessary to insure that 

det( C <^-/>- lB " A ')=0, i = 1,2, ...,n. 

Now, if CB is invertible then we can use the relationship Cx = C Ax + CBu 
together with y = Cx to write y = CAx + CBu. This lets us solve for u and recast the 
system as 

x = {A- B{CB)- l CA)x + B(CB)- 1 y, 
u = {CB)- l y-{CB)- 1 CAx. 

Here we have a set of equations in which the roles of u and y are reversed. They show 
how a choice of y determines x and how x determines u l39l . 

2.3 Stability and Boundedness 

Let a time-varying dynamical system may be expressed as 

x(t) = f(t,x(t)) (5) 

where x £ 1" is an nD vector and / : R+ x D -> R n with D = R n or D = B h for 
some h > 0, where fij = {1 £ K" : \x\ < h} is a ball centered at the origin with a 
radius of h.lfD = R n then we say that the dynamics of the system are defined globally, 
whereas if D = Bh they are only defined locally. We do not consider systems whose 
dynamics are defined over disjoint subspaces of R. It is assumed that f(t,x) is piecemeal 
continuous in t and Lipschitz in x for existence and uniqueness of state solutions. As an 
example, the linear system x(t) = Ax(t) fits the form of ® with D = W 1 [38]. 
Assume that for every xq the initial value problem 

x(t) = f(t,x(t)), x(t ) = x , 

possesses a unique solution x(t, to, x$)\ it is called a solution to (O if x(t, to,xo) = xq 
and £x(t,t ,x ) = f(t,x(t,t ,x )) EH- 

A point x e G W 1 is called an equilibrium point of © if f(t, x e ) = for all t > 0. 
An equilibrium point x e is called an isolated equilibrium point if there exists an p > 
such that the ball around x e , B p (x e ) = {x £ R n : |x — x e | < p}, contains no other 
equilibrium points besides x e ll38l . 

The equilibrium x e = of ([5]> is said to be stable in the sense of Lyapunov if for 
every e > and any to > there exists a 5(e,to) > such that \x(t,to,xo)\ < e for 
all t > to whenever \xq\ < 5(e,to) and x(t, to,xo) 6 Bh(x e ) for some /i > 0. That 
is, the equilibrium is stable if when the system © starts close to x e , then it will stay 
close to it. Note that stability is a property of an equilibrium, not a system. A system is 
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stable if all its equilibrium points are stable. Stability in the sense of Lyapunov is a local 
property. Also, notice that the definition of stability is for a single equilibrium x e G W 1 
but actually such an equilibrium is a trajectory of points that satisfy the differential 
equation in ((5]). That is, the equilibrium x e is a solution to the differential equation (O, 
x(t, to,xo) = x e for t > 0. We call any set such that when the initial condition of ([5]> 
starts in the set and stays in the set for all t > 0, an invariant set. As an example, if 
x e = is an equilibrium, then the set containing only the point x e is an invariant set, 
for © 1551 . 

If 5 is independent of to, that is, if 5 = 5(e), then the equilibrium x e is said to be 
uniformly stable. If in ([5]) / does not depend on time (i.e., f(x)), then x e being stable is 
equivalent to it being uniformly stable. Uniform stability is also a local property. 

The equilibrium x e = of (O is said to be asymptotically stable if it is stable and 
for every to > there exists rj(to) > such that Hindoo \x(t,to, xq)\ = whenever 
\xq\ < r)(to). That is, it is asymptotically stable if when it starts close to the equilibrium 
it will converge to it. Asymptotic stability is also a local property. It is a stronger 
stability property since it requires that the solutions to the ordinary differential equation 
converge to zero in addition to what is required for stability in the sense of Lyapunov. 

The equilibrium x e = of §5$ is said to be uniformly asymptotically stable if it is 
uniformly stable and for every e > and and to > 0, there exist a So > independent 
of to an d e, and a T(e) > independent of to, such that \x(t, to, xq) — x e \ < e for 
all t > to + T(e) whenever |a?o — x e | < 5(e). Again, if in ((U / does not depend on 
time (i.e., f(x)), then x e being asymptotically stable is equivalent to it being uniformly 
asymptotically stable. Uniform asymptotic stability is also a local property. 

The set C M. n of all xq € W 1 such that \x(t, to, Xo)\ — ► as t — ► oo is called 
the domain of attraction of the equilibrium x e = of ((5]). The equilibrium x e = 
is said to be asymptotically stable in the large if C W 1 . That is, an equilibrium is 
asymptotically stable in the large if no matter where the system starts, its state converges 
to the equilibrium asymptotically. This is a global property as opposed to the earlier 
stability definitions that characterized local properties. This means that for asymptotic 
stability in the large, the local property of asymptotic stability holds for Bh(x e ) with 
h = oo (i.e., on the whole state-space). 

The equilibrium x e = is said to be exponentially stable if there exists an a > and 
for every e > there exists a 5(e) > such that \x(t, to, a?o)| < ee~ a ^~ to \ whenever 
|xo| < 5(e) and t > to > 0. The constant a is sometimes called the rate of convergence. 
Exponential stability is sometimes said to be a 'stronger' form of stability since in its 
presence we know that system trajectories decrease exponentially to zero. It is a local 
property; here is its global version. The equilibrium point x e = is exponentially 
stable in the large if there exists a > and for any j3 > there exists e(f3) > such 
that \x(t,t ,x )\ < e(f3)e~ a ^- to \ whenever \x \ < (3 and t > t > 0. 

An equilibrium that is not stable is called unstable. 

Closely related to stability is the concept of boundedness, which is, however, a 
global property of a system in the sense that it applies to trajectories (solutions) of the 
system that can be defined over all of the state-space [38 ]. 

A solution x(t,to,Xo) of © is bounded if there exists a (3 > 0, that may depend 
on each solution, such that \x(t,to,xo)\ < (3 for all t > to > 0. A system is said to 
possess Lagrange stability if for each to > and xo 6 M. n , the solution x(t, to,xo) is 
bounded. If an equilibrium is asymptotically stable in the large or exponentially stable 
in the large then the system for which the equilibrium is defined is also Lagrange stable 
(but not necessarily vice versa). Also, if an equilibrium is stable, it does not imply that 
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the system for which the equilibrium is defined is Lagrange stable since there may be 
a way to pick xq such that it is near an unstable equilibrium and x(t, to, xo) — ► oo as 

t — > oo. 

The solutions x(t, to, xq) are uniformly bounded if for any a > and to > 0, there 
exists a /3(a) > (independent of to) such that if |xo| < a, then \x(t, to, xq)\ < /3(a) 
for all t > to > 0. If the solutions are uniformly bounded then they are bounded and 
the system is Lagrange stable. 

The solutions x(t, to, xo) are said to be uniformly ultimately bounded if there exists 
some B > 0, and if corresponding to any a > and to > there exists a T(a) > 
(independent of to) such that \xo\ < a implies that \x(t,to,Xo)\ < B for all t > 
to + T[a). Hence, a system is said to be uniformly ultimately bounded if eventually all 
trajectories end up in a B— neighborhood of the origin. 



2.4 Lyapunov's Stability Method 

A. M. Lyapunov invented two methods to analyze stability [38]. In his indirect method 
he showed that if we linearize a system about an equilibrium point, certain conclusions 
about local stability properties can be made (e.g., if the eigenvalues of the linearized 
system are in the left half plane then the equilibrium is stable but if one is in the right 
half plane it is unstable). 

In his direct method the stability results for an equilibrium x e = of © depend 
on the existence of an appropriate Lyapunov function V : D — > R where D = W 1 for 
global results (e.g., asymptotic stability in the large) and D = Bh for some h > 0, for 
local results (e.g., stability in the sense of Lyapunov or asymptotic stability). If V is 
continuously differentiable with respect to its arguments then the derivative of V with 
respect to t along the solutions of ((5]> is 

dV dV 

As an example, suppose that (f5]) is autonomous, and let V (x) is a quadratic form V(x) = 
x T Px where x 6 W 1 and P = P T . Then, V(x) = %f{t,x) = x T Px + x T Px = 
2x T Px Il38l. 

Lyapunov's direct method provides for the following ways to test for stability. The 
first two are strictly for local properties while the last two have local and global versions. 

- Stable: If V (t, x) is continuously differentiable, positive definite, and V(t, x) < 0, 
then x e = is stable. 

- Uniformly stable: If V(t, x) is continuously differentiable, positive definite, de- 
cresceno and V(t, x) < 0, then x e = is uniformly stable. 

- Uniformly asymptotically stable: If V(t, x) is continuously differentiable, posi- 
tive definite, and decrescent, with negative definite V(t,x), then x e = is uniformly 
asymptotically stable (uniformly asymptotically stable in the large if all these properties 
hold globally). 

- Exponentially stable: If there exists a continuously differentiable V(t, x) and 
c, ci , C2 , C3 > such that 

ci \x\ c < V(t, x) < C2 \x\ c , (6) 
V(t,x) < -c 31 \x\ c , (7) 



'A C°-function V(t, x) : M+ x B h -> R(V(t, x) : K+ x M™ -> M) is said to be decrescent if there exists 
a strictly increasing function 7 defined on [0, r) for some r > (defined on [0, 00)) such that V(t, x) < 7(|ie|) 
for all t > and x € Bh for some h > 0. 
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for all x £ Bh and t > 0, then x e = is exponentially stable. If there exists a 
continuously differentiable function V(t, x) and Equations © and (0 hold for some 
c, ci , C2 , C3 > for all x G K" and t > 0, then x e = is exponentially stable in the 
large 61. 

2.5 Nonlinear and Impulse Dynamics of Complex Plants 

In this section we give two examples of nonlinear dynamical systems that are beyond 
reach of the classical control theory. 

2.5.1 Hybrid Dynamical Systems of Variable Structure 

Consider a hybrid dynamical system of variable structure, given by n— dimensional 
ODE (see EH) 

x = f(t,x), (8) 

where x = x(t) G R n and / = f(t, x) :l + xR™^ W 1 . Let the domain Gcl+x W\ 
on which the vector-field f(t,x) is defined, be divided into two subdomains, G + and 
G~, by means of a smooth (n— 1)— manifold M. In G + UM, let there be given a vector- 
field f + (t, x), and in G~ U M, let there be given a vector-field f~(t, x). Assume that 
both / + = f + (t, x) and /~ = f~(t, x) are continuous in t and smooth in x. For the 
system ®, let 

j f + when x G G + 
* ~ \ f~ when x G G~ 

Under these conditions, a solution x(t) of ODE ([U) is well-defined while passing through 
G until the manifold M is reached. 

Upon reaching the manifold M, in physical systems with inertia, the transition 

from x = f~(t,x) to x = f + (t, x) 

does not take place instantly on reaching M, but after some delay. Due to this delay, 
the solution x(t) oscillates about M, x(t) being displaced along M with some mean 
velocity. 

As the delay tends to zero, the limiting motion and velocity along M are determined 
by the linear homotopy ODE 

x = f°(t, x) = {\- a) f~{t, x)+a f + (t, x), (9) 

where x 6 M and a £ [0, 1] is such that the linear homotopy segment f°(t,x) is 
tangential to M at the point x, i.e., f°(t, x) G T X M, where T X M is the tangent space to 
the manifold M at the point x. 

The vector-field f°(t,x) of the system (O can be constructed as follows: at the 
point x G M, f~(t,x) and f + (t,x) are given and their ends are joined by the linear 
homotopy segment. The point of intersection between this segment and T X M is the end 
of the required vector-field f°(t, x). The vector function x(t) which satisfies ((U) in G~ 
and G + , and (O when x G M, can be considered as a solution of ([U) in a general sense. 

However, there are cases in which the solution x(t) cannot consist of a finite or even 
countable number of arcs, each of which passes through G~ or G + satisfying ([8]>, or 
moves along the manifold M and satisfies the homotopic ODE (O. To cover such cases, 
assume that the vector-field / = f(t, x) in ODE (d is a Lebesgue-measurable function 
in a domain G C M + x W 1 , and that for any closed bounded domain D C G there exists 
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a summable function K (t) such that almost everywhere in D we have \ f(t, x)\ < K(t). 
Then the absolutely continuous vector function x(t) is called the generalized solution 
of the ODE ((8]) in the sense ofFilippov (see fl46l ) if for almost all t, the vector x = x(t) 
belongs to the least convex closed set containing all the limiting values of the vector 
field f(t,x*), where x* tends towards x in an arbitrary manner, and the values of the 
function f(t, x*) on a set of measure zero in W 1 are ignored. 

Such hybrid systems of variable structure occur in the study of nonlinear electric 
networks (endowed with electronic switches, relays, diodes, rectifiers, etc.), in models 
of both natural and artificial neural networks, as well as in feedback control systems 
(usually with continuous-time plants and digital controllers/filters). 

2.5.2 Impulse Dynamics of Kicks and Spikes 

The Spike Function. Recall that the Dirac's 5— function (also called the impulse 
function in the systems and signals theory) represents a limit of the Gaussian bell- 
shaped curve 

1 2 

g(t,a) = . e~* * a (with parameter a —> 0) (10) 

() where the factor 1/^/ttq. serves for the normalization of (fTOl) . 



+°° dt 



e"* 2 / Q = l, (11) 



i.e., the area under the pulse is equal to unity. In (flOl . the smaller a the higher the peak. 
In other words, 

S(t) = lim -Le" i2/a , (12) 



which is a pulse so short that outside of t = it vanishes, whereas at t = it still 
remains normalized according to (TTTb . Therefore, we get the usual definition of the 
5— function: 

S(t) = for t / 0, 
S(t)dt = 1, (13) 

where e may be arbitrarily small. Instead of centering the 5— pulse around t = 0, we 
can center it around any other time to so that (TT3T ) is transformed into 

5(t-t ) = for t^t , 

t +e 

8(t-t )dt = 1. (14) 

t -e 



Another well-known fact is that the integral of the 5— function is the Heaviside's 
step function 

for T < 

1 for T>0 . (15) 

7 for T = 0) 

Now we can perform several generalizations of the relation ( PT5T ). First, we have 



f T ( 

H(T) = / 5{t) dt= I 1 

J-oo [ (1 



T 

5(ct - t Q ) dt 



for T < t /c 
1/c for T > t /c 
± for T = t /c 
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More generally, we can introduce the so-called phase function (f>(t), (e.g., <j)(t) = ct — 
to) which is continuous at t = to but its time derivative <j>(t) = ^j^- is discontinuous at 
t = to (yet positive, cf)(t) > 0), and such that 



T 

5{<j>{t)) dt = { 



for T < t 
l/0(*o) for T>t 

jk f ° r T = t ° 



Finally, we come the the spike function 5{4>{t))4>{t), which like 5— function represents 
a spike at t = to, such that the normalization criterion (fl~4l) is still valid, 

t +e 

6(<p(t))<j>(t) dt = 1. 

to— e 

Deterministic Delayed Kicks. Following Haken [47 ], we consider the mechanical 
example of a soccer ball that is kicked by a soccer player and rolls over grass, whereby 
its motion will be slowed down. In our opinion, this is a perfect model for all 'shooting- 
like' actions of the human operator. 

We start with the Newton's (second) law of motion, mv = force, and in order to 
get rid of superfluous constants, we put temporarily m = 1. The force on the r.h.s. 
consists of the damping force —jv(t) of the grass (where 7 is the damping constant) 
and the sharp force F(t) = sS(t — a) of the individual kick occurring at time t = a 
(where s is the strength of the kick, and S is the Dirac's 'delta' function). In this way, 
the single-kick equation of the ball motion becomes 

v = -jv(t) + s5(t-a), (16) 

with the general solution 

v (t) = sG(t - a), 
where G(t — a) is the Green's functional 



G(t - a) 



for t < a 

e -7(i-0 f or t > a ■ 



Now, we can generalize the above to N kicks with individual strengths Sj, occurring 
at a sequence of times {<Jj}, so that the total kicking force becomes 

N 

F(t) = J2s j S(t-a j ). 



2 This is the Green's function of the first order system ([Tol l. Similarly, the Green's function 

f for t < a 

G[t - a >-\ (t- «r) e -T(*-") for t > a 



corresponds to the second order system 

* + 7) G(t-a)=S{t-a). 



d ^ 2 



dt 
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In this way, we get the multi-kick equation of the ball motion 

N 

V = —jv(t) + Sj5(t — (Tj), 
3=1 

with the general solution 

N 

v(t) = J2s j G(t-a j ). (17) 

3=1 

As a final generalization, we would imagine that the kicks are continuously exerted 
on the ball, so that kicking force becomes 

F(t)= f s{a)5{t-a)da= f daF(a)S(t - a), 

J t(, J to 

so that the continuous multi-kick equation of the ball motion becomes 

v = - 7 u(t) + I s(a)5(t - a)do = --yv(t) + f daF(a)S(t - a), 

J to J to 

with the general solution 

i-T r T 



v(t)= ! daF{a)G{t- a) = [ daF{a)e~^ 
ho ho 



e-^- a >. (18) 



Random Kicks and Langevin Equations. We now denote the times at which 
kicks occur by tj and indicate their direction in a one-dimensional game by (±l)j, 
where the choice of the plus or minus sign is random (e.g., throwing a coin). Thus the 
kicking force can be written in the form 

N 

F(t) = sY,Ht-t j )(± 1 )j, (19) 

3=1 

where for simplicity we assume that all kicks have the same strength s. When we 
observe many games, then we may perform an average < ... > over all these different 
performances, 

N 

< F{t) >=s<J2 S (t- tj)^ 1 )] > • ( 20 ) 

3=1 

Since the direction of the kicks is assumed to be independent of the time at which the 
kicks happen, we may split (l20l ) into the product 

N 

< F(t) >= s < ^ <5(i - tj) >< (±l)j > . 

3=1 

As the kicks are assumed to happen with equal frequency in both directions, we get the 
cancellation 

< (±1), >= 0, 
which implies that the average kicking force also vanishes, 

< F(t) >= 0. 
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In order to characterize the strength of the force ( fT9l ), we consider a quadratic expression 
in F, e.g., by calculating the correlation function for two times t, t', 



<F(t)F(t') >=s 2 < £tf(t-t i )(±l) J -£5(t / -t fc )(±l)k > • 

j k 

As the ones for j ^ k will cancel each other and for j = k will become 1, the correlation 
function becomes a single sum 

< F(t)F(t') >= s 2 < S(t - tj)5(t' - t k ) >, (21) 

3 

which is usually evaluated by assuming the Poisson process for the times of the kicks. 

Now, proper description of random motion is given by Langevin rate equation, 
which describes the Brownian motion: when a particle is immersed in a fluid, the veloc- 
ity of this particle is slowed down by a force proportional to its velocity and the particle 
undergoes a zig-zag motion (the particle is steadily pushed by much smaller particles 
of the liquid in a random way). In physical terminology, we deal with the behavior of a 
system (particle) which is coupled to a heat bath or reservoir (namely the liquid). The 
heat bath has two effects: 

1. It decelerates the mean motion of the particle; and 

2. It causes statistical fluctuation. 

The standard Langevin equation has the form 

v = - 7 v(t) + F(t), (22) 

where F(t) is a. fluctuating force with the following properties: 

1. Its statistical average (l20l ) vanishes; and 

2. Its correlation function (|2TT> is given by 

<F(t)F(t')>=QS(t-to), (23) 

where to = T/N denotes the mean free time between kicks, and Q = s 2 /to is the 
random fluctuation. 



The general solution of the Langevin equation (1221 is given by (fl8l ). 
The average velocity vanishes, < v(t) >= 0, as both directions are possible and 
cancel each other. Using the integral solution (fT8T ) we get 

ft' 

'to Jto 

which, in the steady-state, reduces to 



ft ft' 

< v(t)v(t') >=< da da'F(a)F{a')e-^ t -^e-^ t '-^ >, 

Jtn J tn 



<v(t)v(t>) >=9-e-^-°), 

and for equal times 

< „(<)* >= g. 
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If we now repeat all the steps performed so far with ra/1, the final result reads 

< v{tf >= ® (24) 
27m 

Now, according to thermodynamics, the mean kinetic energy of a particle is given 

- < v(t) 2 >= -k B T, (25) 

where T is the (absolute) temperature, and k B is the Boltzman's constant. Comparing 
(1241 and ( [251 , we obtain the important Einstein's result 

Q = 2jk B T, 

which says that whenever there is damping, i.e., 7 / 0, then there are random fluctua- 
tions (or noise) Q. In other words, fluctuations or noise are inevitable in any physical 
system. For example, in a resistor (with the resistance R) the electric field E fluctuates 
with a correlation function (similar to d23l) ) 



< E(t)E(t') >= 2Rk B T5(t - t ). 
This is the simplest example of the so-called dissipation-fluctuation theorem. 



3 Nonlinear Control Modeling of the 
Human Operator 

In this section we present the basics of modern nonlinear control, as a powerful tool for 
controlling nonlinear dynamical systems. 

3.1 Graphical Techniques for Nonlinear Systems 

Graphical techniques preceded modern geometrical techniques in nonlinear control the- 
ory. They started with simple plotting tools, like the so-called 'tracer plot'. It is a useful 
visualization tool for analysis of second order dynamical systems, which just adds time 
dimension to the standard 2D phase portrait. For example, consider the damped spring 
governed by 

x = —kx — x, x(0) = 1. 

Its tracer plot is given in Figure Q] Note the stable asymptote reached as t — > 00. 

The most important graphical technique is the so-called describing function analy- 
sis. 



3.1.1 Describing Function Analysis 

Describing function analysis extends classical linear control technique, frequency re- 
sponse analysis, for nonlinear systems ll37l . It is an approximate graphical method 
mainly used to predict limit cycles in nonlinear ODEs. 

For example, if we want to predict the existence of limit cycles in the classical Van 
der Pol's oscillator given by 

x + a(x 2 - 1) x + x = 0, (26) 
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Trajectory Plot 




1 -0.5 



time 



Figure 1: Tracer plot of the damped spring. 

Nonlinear element Linear element 




(zero) 



Figure 2: Feedback interpretation of the Van Der Pol oscillator (after D7I0 . Here p is a 
(linear) differentiator, and II a (nonlinear) multiplicator. 



we need to rewrite (1261 ) as a linear unstable low-pass block and a nonlinear block (see 
FigurelU). In this way, using the nonlinear block substitution, w := —xx 2 , we get 

x — ax + x = aw, or 
x(p 2 — ap + 1) = aw, 

or just considering the transfer function from w to x, 

x a 



w p 2 — ap + 1 

Now, if we assume that the Van der Pol oscillator does have a limit cycle with a fre- 
quency of 

x(t) = Asm(wt), 
so x = Aw cos(wt), therefore the output of the nonlinear block is 

z = — x 2 x = —A 2 sin 2 (wt) Aw cos(wt) 
A^w 

= — (1 — cos(2wt)) cos(wt) 

= — (cos(wt) — cos(3wt)) . 
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Note how z contains a third harmonic term, but this is attenuated by the low-pass na- 
ture of the linear block, and so does not effect the signal in the feedback. So we can 
approximate z by 

A 3 A 2 d 

z — — wcos(wt) = — — — (— A sin(wt)) . 
4 K ! 4 dt y y " 

Therefore, the output of the nonlinear block can be approximated by the quasi-linear 
transfer function which depends on the signal amplitude, A, as well as frequency. The 
frequency response function of the quasi-linear element is obtained by substituting p = 

s = iw, 

A 2 

N(A,w) = —(iw). 
Since the system is assumed to contain a sinusoidal oscillation, 

x = Asin(wt) = G(iw) z 
= G(iw)N(A,w)(-x), 

where G(iw) is the transfer function of the linear block. This implies that, 

— = -1 = G(iw)N(A,w), 
—x 

so 

1 ] A2 ( iw ) a _ Q 

4 (iw) 2 — a(iw) + 1 

which solving gives, 

A = 2, uj = \, 

which is independent of a. Note that in terms of the Laplace variable p = s, the closed 
loop characteristic equation of the system is 

A 2 (iw) a 
4 p z — ap + 1 

whose eigenvalues are 



1 ,,2 x a 2 (A 2 -A) 2 

Corresponding to A = 2 gives eigenvalues of Ai 2 = indicating an existence of 
a limit cycle of amplitude 2 and frequency 1 (see Figure [3]). If A > 2 eigenvalues 
are negative real, so stable, and the same holds for A < 2. The approximation of the 
nonlinear block with (A2/A)(iw) is called the describing function. This technique is 
useful because most limit cycles are approximately sinusoidal and most linear elements 
are low-pass in nature. So most of the higher harmonics, if they existed, are attenuated 
and lost. 



3.2 Feedback Linearization 

The idea of feedback linearization is to algebraically transform the nonlinear system 
dynamics into a fully or partly linear one so that the linear control techniques can be 
applied. Note that this is not the same as a conventional linearization using Jacobians. 
In this subsection we will present the modern, geometrical, Lie-derivative based tech- 
niques for exact feedback linearization of nonlinear control systems. 
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Figure 3: Limit cycle of the Van der Pol oscillator (after Il37l0 . Approximation is reasonable 
for small a, but error, (amplitude should be equal to 2), grows as a is increased. 



3.2.1 The Lie Derivative and Lie Bracket in Control Theory 

Recall that given a scalar function h(x) and a vector-field f(x), we define a new scalar 
function, Cfh := V/i/, which is the Lie derivative of h w.r.t. /, i.e., the directional 
derivative of h along the direction of the vector / (see El HI). Repeated Lie derivatives 
can be defined recursively: 

C° f h = h, 

C\h = C f (Ci^h) = V (c^h) f, for i = 1, 2, ... 



Or given another vector-field, g, then C g Cfh(x) is defined as 

Cg£ f h = V(C f h)g. 
For example, if we have a control system 

x = f(x), 

y = h(x), 

with the state x = x(t) and the the output y, then the derivatives of the output are: 

y = —x = Cfh, and 
ox 

dLfh 

v = -ih x = c s h - 

ox 1 

Also, recall that the curvature of two vector-fields, g\,g2, gives a non-zero Lie 
bracket, [51,52] (see Figure [4]). Lie bracket motions can generate new directions in 
which the system can move. 
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Figure 4: 'Lie bracket motion' is possible by appropriately modulating the control inputs 
(from 021). 

In general, the Lie bracket of two vector-fields, f(x) and g(x), is defined by 

[f, g] := ad f g := Vgf - Vfg : = ||/ - |4, 

where V/ := df/dx is the Jacobian matrix. We can define Lie brackets recursively, 

ad° f g = g, 

ad)g = \J,cu#f l g\, fori = 1,2,... 

Lie brackets have the properties of bilinearity, skew-commutativity and Jacobi identity. 
For example, if 



then we have 

[/,<?] 



I COS X2 \ Xl 
f={ *1 h 9= { 1 



1 \ / cos xi \ f — sinx2 \ f x\ 
){ x t ){ 1 

cos X2 + sin X2 

-Xl 

3.2.2 Input/Output Linearization 

Given the single-input single-output (SISO) system 

x = f{x)+g(x)u, (27) 

we want to formulate a linear differential equation relation between output y and a new 
input v. We will investigate (see BTll43ll37l '): 

• How to generate a linear input/output relation. 

• What are the internal dynamics and zero-dynamics associated with the input/output 
linearization? 

• How to design stable controllers based on the FO linearization. 

This linearization method will be exact in a finite domain, rather than tangent as 
in the local linearization methods, which use Taylor series approximation. Nonlinear 
controller design using the technique is called exact feedback linearization. 
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3.2.3 Algorithm for Exact Feedback Linearization 

We want to find a nonlinear compensator such that the closed-loop system is linear 
(see Figure [5]). We will consider only affine SISO systems of the type (127T ). i.e, x = 
f(x) + g(x) u, y = h(x), and we will try to construct a control law of the form 



u = p(x) + q(x) v, 

where v is the setpoint, such that the closed-loop system 

x = f(x)+g(x)p(x)+g(x)q(x)v, 
V = h(x), 



(28) 



is linear from command v to y. 



comand, r 



controller 



plant 




state feedback vector, x 

Linear 'black box' 



Figure 5: Feedback linearization. 

The main idea behind the feedback linearization construction is to find a nonlinear 
change of coordinates which transforms the original system into one which is linear and 
controllable, in particular, a chain of integrators. The difficulty is finding the output 
function h{x) which makes this construction possible. 

We want to design an exact nonlinear feedback controller. Given the nonlinear affine 
system, x = f(x) + g(x), y = h(x), we want to find the controller functions p(x) and 
q{x). The unknown functions inside our controller (l28l are given by: 



p(x) 
q(x) 



C r f h{x) + faPfi-hix) + ... + r - X Cfh(x) + r h(s 



c g c r f - l h{ x y 



(29) 



which are comprised of Lie derivatives, Cfh(x). Here, the relative order, r, is the 
smallest integer r such that £ 3 £y _1 /i(x) ^ 0. For linear systems r is the difference 
between the number of poles and zeros. 

To obtain the desired response, we choose the r parameters in the (5 polynomial to 
describe how the output will respond to the setpoint, v (pole-placement). 

dT y > a drly , . a dy , a 
Here is the proposed algorithm fiTll43ll37l ): 

1. Given nonlinear SISO process, x = f(x, u), and output equation y = h(x), then: 



21 



2. Calculate the relative order, r. 

3. Choose an rth order desired linear response using pole-placement technique (i.e., 
select f3). For this could be used a simple rth order low-pass filter such as a 
Butterworth filter. 

4. Construct the exact linearized nonlinear controller (l29l ). using Lie derivatives and 
perhaps a symbolic manipulator (Mathematica or Maple). 

5. Close the loop and obtain a linear input-output black-box (see Figure[5]>. 

6. Verify that the result is actually linear by comparing with the desired response. 

3.3 Controllability 

3.3.1 Linear Controllability 

A system is controllable if the set of all states it can reach from initial state xq = x(0) at 
the fixed time t = T contains a ball B around xq. Again, a system is small time locally 
controllable (STLC) iff the ball B for t < T contains a neighborhood of xoll 
In the case of a linear system in the standard state-space form 

x = Ax + Bu, (30) 

where A is the n x n state matrix and B is the m x n input matrix, all controllability 
definitions coincide, i.e., 

- x(T), 
x(0) 0, 
x(0) - x(T), 

where T is either fixed or free. 

Rank condition states: System (l30l is controllable iff the matrix 

W n = (BAB... A"- l B) has full rank. 

In the case of nonlinear systems the corresponding result is obtained using the for- 
malism of Lie brackets, as Lie algebra is to nonlinear systems as matrix algebra is to 
linear systems. 

3.3.2 Nonlinear Controllability 

Nonlinear MIMO-systems are generally described by differential equations of the form 
(see ED El EH): 

x = fix) + gi(x)u\ (i = l,...,n), (31) 

defined on a smooth n— manifold M, where x E M represents the state of the control 
system, f(x) and gi(x) are vector-fields on M and the u l are control inputs, which 
belong to a set of admissible controls, u l G U. The system (I3TT) is called driftless, or 
kinematic, or control linear if f(x) is identically zero; otherwise, it is called a system 
with drift, and the vector-field f(x) is called the drift term. The flow 4> 9 t (xq) represents 
the solution of the differential equation x = g(x) at time t starting from xq. Geometrical 
way to understand the controllability of the system (I3TT) is to understand the geometry 
of the vector-fields f(x) and gi(x). 

3 The above definition of conttollability tells us only whether or not something can reach an open neighbor- 
hood of its starting point, but does not tell us how to do it. That is the point of the trajectory generation. 
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Example: Car-Parking Using Lie Brackets. In this popular example, the driver 
has two different transformations at his disposal. He can turn the steering wheel, or he 
can drive the car forward or back. Here, we specify the state of a car by four coordinates: 
the (x, y) coordinates of the center of the rear axle, the direction 9 of the car, and the 
angle (p between the front wheels and the direction of the car. L is the constant length 
of the car. Therefore, the configuration manifold of the car is 4D, M := (x, y, 9, <fr). 
Using (PTT ). the driftless car kinematics can be defined as: 



x = gi{x)ui + g 2 (x) u 2 , 

with two vector-fields g\,g 2 G X k (M) . 

The infinitesimal transformations will be the vector-fields 



. . „3 . „3 tan0 d 

g x (x) = drive = cos 6— + sin 9— + — — — 
ox ay L o9 



(32) 



/ cos 9 
sm.9 
■j tan( 




V 



/ 



and 



g 2 {x) = STEER 



d_ 

50 






V 1 / 



Now, STEER and DRIVE do not commute; otherwise we could do all your steering 
at home before driving of on a trip. Therefore, we have a Lie bracket 



[52,5l] = [STEER, DRIVE] 



1 







L cos 2 6 89 



ROTATE. 



The operation [g 2 ,gi] = ROTATE = [steer, drive] is the infinitesimal version of the 
sequence of transformations: steer, drive, steer back, and drive back, i.e., 



{STEER, DRIVE, STEER , DRIVE }. 

Now, ROTATE can get us out of some parking spaces, but not tight ones: we may not 
have enough room to ROTATE out. The usual tight parking space restricts the DRIVE 
transformation, but not STEER. A truly tight parking space restricts STEER as well by 
putting your front wheels against the curb. 

Fortunately, there is still another commutator available: 



[gi, [52,51]] 
[drive, rotate] 



[drive, [steer, drive]] = [[51, 52], 51] 



Lcos 2 



8111 ( 



d_ 

dx 



cos ( 





1 ____ 

dy 



SLIDE. 



The operation [[gi , g 2 ] , gi] = SLIDE = [drive,ROTATe] is a displacement at right angles 
to the car, and can get us out of any parking place. We just need to remember to steer, 
drive, steer back, drive some more, steer, drive back, steer back, and drive back: 

{STEER, DRIVE, STEER -1 , DRIVE, STEER, DRIVE -1 , STEER" 1 , DRIVE -1 }. 

We have to reverse steer in the middle of the parking place. This is not intuitive, and no 
doubt is part of the problem with parallel parking. 
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Thus from only two controls u\ and u 2 we can form the vector fields DRIVE = 
gi, STEER = 52, ROTATE = [g2,gi], and SLIDE = [[ffi,32])5i] s allowing us to move 
anywhere in the configuration manifold M. The car kinematics x = 5i«i + 52^2 is 
thus expanded as: 



/ x \ 

y 
9 



DRIVE • U\ + STEER • 1i 2 



( cos e \ 

sm.6 
■£ tan</> 
V J 



■ ui + 






V 1 / 



• u 2 ■ 



The parking theorem says: One can get out of any parking lot that is larger than the 



car. 




(xi,x 2 ) 



Figure 6: The unicycle. 



The Unicycle Example. Now, consider the unicycle example (see Figure ©. Here 
we have 



COS £3 ^ 






sin #3 


|. H 




) 







/ sin x 3 \ 
[51,52] = -COSX3 . 

V J 

The unicycle system is full rank and therefore controllable. 
3.3.3 Controllability Condition 

Nonlinear controllability is an extension of linear controllability. The nonlinear SIMO 
system 

x = f(x) +g{x)u 
is controllable if the set of vector-fields 

{<?,[/,<?],•••, IT- 1 ,*?]} 

is independent. 

For example, for the kinematic kar system of the form (l32l ). the nonlinear controlla- 
bility criterion reads: If the Lie bracket tree: 

91, 92, bi,52], [[51, 52], 5i], [[51, 52], 52], [[[51, 52], 5i], 5i], [[[51, 52], 5i], 52], 
[[[51, 52], 52], 5i], [[[51, 52], 52], 52], ••• 
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- has full rank then the system is controllable fl4Tll44ll42ll . In this case the combined 
input 

(1,0), t€[0,e] 

(0,1), t€[e,2e] 

(-1,0), t£[2e,3e] 

{ (0,-1), t £ [3e,4e] 

gives the motion x{Ae) = x(0) + e 2 [51, 52] + 0{e i ), with the flow given by 



(ui,u 2 ) 



pjgim] _ lim [ ^-32 _p-Si ^92 ^Sl 



n— >oo V y/t/n yjt/n yjt/n \/t/n 
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3.4 Adaptive Lie-Derivative Control 

In this subsection we develop the concept of machine learning in the framework of Lie- 
derivative control formalism (see (13.2.11) above). Consider an n— dimensional, SISO 
system in the standard affine form d27l . rewritten here for convenience: 



x(t) = f(x) + g(x) u(t), y(t) = h{x), (33) 



As already stated, the feedback control law for the system (1331) can be defined using 
Lie derivatives Cfh and C g h of the system's output h along the vector-fields / and g. 

If the SISO system (l33l is a relatively simple (quasilinear) system with relative 
degreJ^ = 1, it can be rewritten in a quasilinear form 

x{t)= 7i {t)f i (x)+d 1 (t)g 1 (x)u{t), (34) 

where 7» (i = 1, ...,n) and dj (j = 1, ...,m) are system's parameters, while f\ and #j 
are smooth vector-fields. 

In this case the feedback control law for tracking the reference signal yn = yii(t) is 
defined as (see I4T1I441 ) 

„, _ ~ c f h + VR + a {y R - y) (35) 



Cgll 

where a denotes the feedback gain. 

Obviously, the problem of reference signal tracking is relatively simple and straight- 
forward if we know all the system's parameters ji(t) and dj(t) of (f34b - The question is 
can we apply a similar control law if the system parameters are unknown? 

Now we have much harder problem of adaptive signal tracking. However, it appears 
that the feedback control law can be actually cast in a similar form (see 1431 .1450: 



-Cfh + yn + a (y R - y) 



Cgh 



where Lie derivatives Cfh and C g h of (1351) have been replaced by their estimates Cfh 
and Cgh, defined respectively as 

Cfh = %(t)Cf i h, Cgh = dj(t) C 9i h, 



4 Relative degree equals the number of differentiations of the output function y required to have the input u 
appear explicitly. Technically, the system d33T l is said to have relative degree r at the point x° if (see 114111441 ') 

(i) £g£jh(x) = for all ir in a neighborhood of x° and all k < r — 1, and 

(ii) c g jcr f - l h(3p)^o, 

where Cjh denotes the fcth Lie derivative of h along /. 



25 



in which %{t) and dj(t) are the estimates for 7,(i) and dj(t). 

Therefore, we have the straightforward control law even in the uncertain case, pro- 
vided that we are able to estimate the unknown system parameters. Probably the best 
known parameter update law is based on the so-called Lyapunov criterion (see 11431 ) 
and given by 

■0 = - 7 e W, (37) 



where ip = {7^ — %, dj — dj} is the parameter estimation error, e = y ■ 
error, and 7 is a positive constant, while the matrix W is defined as: 



yjl is the output 



ir 



[w?w?f, 



with 



C 9l h 



-Cfh + i) R + a(y R -y) 
C g h 



The proposed adaptive control formalism (13614371 ) can be efficiently applied wher- 
ever we have a problem of tracking a given signal with an output of a SISO-system 
(I33~l - l34l with unknown parameters. 



4 Conclusion 

In this paper we have presented two approaches to the human operator modeling: linear 
control theory approach and nonlinear control theory approach, based on the fixed and 
adaptive versions of a single-input single output Lie-Derivative controller. Our future 
work will focus on the generalization of the adaptive Lie-Derivative controller to MIMO 
systems. It would give us a rigorous closed-form model for model-free neural networks. 
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